WO 00/58698 



£09/ 



9/937188 

21 SEP 2001 



PCT/GB00/01127 



COLOUR MONITORING 



The invention relates to colour monitoring. 



5 PCT application WO 96/05489 describes a colour 

inspection system and method by which a video camera is 
controlled for the acquisition of a colour image under 
varying image capture conditions . A fast moving 

(approximately 500 metres min* 1 ) web of printed material 

10 passing in front of the camera is monitored. The video 
camera of such a system has a red, green and blue output 
and is provided with a controllable iris aperture, a 
controllable overall RGB signal gain, independently 
controllable RGB signal channel gain, or a controllable RGB 

15 signal channel balance. The controllable parameters are 
controlled by a control unit in the form of a personal 
computer or other suitable processor having an image 
capture board. The processor generates signals from an 
image signal received from the video camera, the signals 

2 0 being used to correct the controllable parameters to 

improve camera image capture performance. 

The subject of the present application comprises an 
improvement to the system and methods described in the 
25 aforementioned PCT application. 

A first problem of the prior system and colour 
monitoring systems in general to be addressed by 
embodiments of the present invention relates to the 

3 0 correction of camera output when used to monitor colours. 

In an ideal world, the video camera characteristic in terms 
of RGB output would be a linear characteristic so that zero 
light intensity from a completely black scene would give 
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zero voltage outputs from a camera. Again, in an ideal 
world, the RGB outputs would increase linearly accordingly 
to increasing light intensity for a particular given 
colour. Unfortunately, real cameras exhibit an imperfect 

5 characteristic dependent upon ambient temperature giving an 
offset from the origin (i.e. a non-zero volt output for 
zero light intensity input) , after which there follows a 
generally linear response region which then, above a 
particular light intensity will tend to flatten out to give 

0 a non- linear response. 

A second problem of the prior system to be addressed by 
embodiments of the present invention concerns the effects 
of non-uniformity of lighting characteristics across a 
5 material to be monitored. Again, in an ideal world, an 
unprinted stationary (or moving) material of uniform colour 
and consistency when monitored by a video camera would 
•appear as such. However, where colour monitoring is to be 
carried out, non-uniform lighting can cause the camera 
monitor to perceive different parts of the camera field of 
view (which typically may be of an A4 -sized area) to have a 
different colour or different lightness value. Ambient 
lighting by its nature is non-uniform both spatially and 
temporally and is therefore inappropriate for colour 
monitoring. A pure flash lighting of the web during colour 
monitoring procedures is generally used but even this has 
problems due to reflections within the lighting 
(monitoring) enclosure and flash to flash intensity 
variations. Also, the physical characteristics of the 
monitored material itself and the variation in angle of 
lighting on the material across the field of view means 
that different parts will reflect or transmit incident 
light to different extents, and this is further dependent 
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on where the lighting source is positioned. In other 
words, the material will not only reflect incident light 
but also transmit it to a degree and, since it is reflected 
light which a camera with a front -mounted lighting source 

5 actually monitors (which will include reflections from the 
monitoring enclosure itself subsequently passing again 
through the translucent .material) , a uniform coloured piece 
of material may be interpreted by a camera as being of non- 
uniform colour/ lightness because of the type and angle of 

0 the light sources, positioning of the camera, type of 
enclosure and composition of the material itself. 

A third problem which has been encountered in colour 
monitoring is that the print design sometimes contains 
5 areas of "half-tone" and "vignettes" - i.e. areas of non- 
uniform colour, which nevertheless need to be checked, for 
consistency through the print run, and in subsequent runs. 
To do this , the same area in the repeat must be inspected 
with high spatial accuracy. In prior systems, triggering 
of image capture and lighting is typically done by 
registration marks located, for instance, at regular 
intervals and to one side of the web. When a registration 
mark is detected by a sensor, both the illuminating flash 
is triggered and image capture carried out in a 
synchronized fashion. This type of triggering is 

advantageous in normal situations in that even if the speed 
of the web is inconsistent, triggering based upon web 
positions will always result in a consistent image capture 
of a particular part of the web. In this context, it must 
be appreciated that in most situations, the printed web is 
likely to be a complex mix of colours which need to be 
accurately monitored, but which also have a given repeat 
length. By carrying out the aforementioned type of 
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triggering with registration marks appropriately positioned 
with respect to the cameras field of view and the repeat 
length, it can be ensured that the colour monitoring is 
always done consistently with regard to the web repeat. 
5 However, using registration marks in the conventional 
manner, the high spatial accuracy required for consistent 
checking of specific areas is not achievable. Neither can 
this conventional type of system cope with print webs which 

do not have a consistent and predictable repeat length. 
10 Also, in high speed web operation, it is not uncommon for 

the web to shift sideways by a few centimetres between 

repeats and this also causes problems with the conventional 

image capture method. 

15 With a view to solving or reducing the first problem 

mentioned above, there is provided a method of calibrating 
a colour monitoring system so as to compensate for non- 
ideal real camera characteristics, the method comprising: 
establishing a camera offset by measuring or calculating 
20 the output voltage of the camera when substantially no 
light falls on any of its sensor elements, hereinafter 
referred to as establishing the offset; establishing the 
point at which a graph of input light intensity against 
camera output voltage starts to deviate from a 
25 substantially linear characteristic, hereinafter referred 
to as establishing the knee; and restricting the amount of 
light incident on all sensor elements of the camera such 
that the maximum output corresponds to a voltage at, or 
below, the knee, and lower light intensities are all within 
30 the range of linear operation. 

The step of establishing of camera offset may be 
carried out on a periodical basis to keep pace with the 
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variations in offset value caused by variation in ambient 
conditions . 

The step of establishing offset may be carried out 
5 whenever an image capture operation for capturing a desired 
image to be monitored is to be carried out . 

Setting the point of zero light intensity may be 
achieved by closing the camera iris. 

.0 

Setting the point of zero light intensity may be 
achieved by setting the camera to monitor a black image, or 
a black part of an image field of the camera. 

5 Setting the point of zero light intensity may be 

achieved by extrapolating from measurements obtained from 
two or more points of known reflectance somewhere in an 
image- field of the camera. 

0 Preferably, one point of known reflectance comprises an 

integral part of the desired image itself. 

Preferably, one point of known reflectance comprises a 
white reference tile within the image field. 

This has the advantage that it may be done for every 
image captured . 



Preferably, there is provided a source of maximum light 
reflectance within the image field by ensuring that a white 
object is present somewhere in the image field. 
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Restricting the camera to operate within the linear 
region may be achieved by reducing the camera aperture bv 
closing the iris to a predetermined degree such that the 
output voltage when measuring the source of maximum light 
5 intensity corresponds to a camera output voltage at or 
below the knee. 

In practice, it is found with preferred embodiments 
that for a typical camera, the iris may be restricted so as 
10 to give a proportionate reduction in the full scale output 
voltage of the camera so as to ensure that a perfect white 
reflector registers at the top of the linear region and to 
then scale down to find appropriate values of camera output 
versus light intensity. 

Preferably, the step of establishing the knee is 
carried out less frequently than the step of establishing 
the offset and may be carried out before commencing and/or 
after completing a plurality of print runs each comprising 
a. plurality of image capture operations. 

With a view to solving or reducing the second problem 
mentioned above, there is provided a method of compensating 
for non- uniformity of lighting across a camera field of 
view when performing an image capture operation to be 
monitored by the camera, the method comprising : 

positioning substantially uncoloured or uniformly 
surface coloured material of known colour characteristics 
in a field of view of the camera; and 

capturing image data from the camera and storing such 
captured image data as a "unif orrr.icy image" in memorv the 
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image data of the uniformity image including information 
concerning the degree of light reflected from the different 
spatial areas of the material detected by the camera. 

By utilizing image data from the camera relating to the 
known coloured material, differences in data from the 
different parts of the field of camera view are 
attributable to the non-uniformity of lighting. Using this 
position related data, subsequently captured image data can 
be adjusted to be corrected for this positional effect. 

In accordance with the method, image capture operations 
of images to be inspected are thereafter carried out and 
compensation for non-uniform lighting conditions effected 
for each captured image. The compensation may comprise 
normalising each newly captured image across the camera 
field of view by determining spatial adjustment factors 
from the uniformity image and, where no uniformity image 
data is available for a particular spatial location, 
interpolating between the various positions or 
extrapolating beyond them. 

The step of normalising image data may comprise the 
sub-steps of: recording output data from the camera 
averaged over all the pixels in a given spatial area of the 
sample with the sample at a first, training, position on 
the printed web to give first position camera channel 
output data C T ; and recording the camera output data from 
the camera averaged over all the pixels in the same given 
spatial area of the sample with the sample at a second 
position on the printed web having different lighting 
conditions to give second position camera channel output 
data C s . 
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The step of normalizing may comprise normalizing the 
second position camera channel output data C s prior to 
comparing them to the trained values from the first 
5 position camera channel output data C T by processing them 
in accordance with the following equation: C s x Ci/C 2/ where 
Ci is the average channel value for the area of the 
unprinted web corresponding to the first position, and C 2 
is the average channel value for the unprinted web 

0 corresponding to the second, differently lighted position. 

Preferably, three camera channels R, G and B (red, 
green, blue) are present such that, for instance, for the 
red channel R, the first position camera channel output 

1 data is R T/ the second position camera channel output data 
is R S/ the average red value for the area of the unprinted 
web corresponding to the starting position is R 2 and the 
average value of the unprinted web corresponding to the 
second, differently lighted, position is R 2 . 

It will be appreciated in the above that the geometry 
of the illumination source and the camera are arranged so 
as to avoid specular reflections in both the "uniformity 
image " and the printed web image . 

Operation must be made in the linear region of the 
camera channel and using the correct current offset. 

Both the method of the first and second aspect, would 
in practice, be carried out for each of the individual 
camera channels (for instance, red (R) , green (G) and blue 
(B) . 
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With a view to solving or reducing the third problem 
mentioned above, there is provided a method for the capture 
and analysis of image data of a moving material monitored 
by a camera having its field of view trained on the 
material, the method comprising: 

storing a pattern recognition model comprising image 
data corresponding to at least part of a pattern repeat 
printed on the monitored material; 

measuring the displacement of the pattern model in each 
captured image of the printed web relative to a captured 
training image to sub-pixel accuracy; and 

calculating the colour of the displaced vignette or 
half-tone area allowing for sub-pixel displacements (both 
vertical and horizontal) . 

Subsequent to the step of measuring the displacement 
and prior to the step of calculating the colour, there may 
be performed the following steps: 

on the basis of the measured displacement, calculating 
a displacement value by which a camera used to capture 
images of the web is to be moved in a transverse direction 
relative to the web so as to substantially reduce 
transverse displacement of the pattern recognition model 
relative to the captured training image during a subsequent 
image capture operation; 

moving said camera accordingly to the calculated 
displacement value; and 
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carrying out said subsequent image capture operation. 

Preferably, following said step of carrying out said 
subsequent image capture operation, there is then carried 
out a further step comprising measuring the displacement of 
the pattern model in the newly captured image of the 
printed web to sub-pixel accuracy. 

Preferably, displacements of the pattern model in the 
longitudinal direction of the web are compensated for by 
delaying or speeding up a trigger signal fed to the camera, 
so as to perform an image capture operation relatively 
earlier or later according to a measured longitudinal 
displacement of the pattern recognition model relative to 
the captured training image. 

The method may further include the steps of: 

monitoring synchronization signals (SYNC) from the 
camera ; and 

in accordance with the operating characteristics of the 
camera, triggering a lighting source during the camera 
vertical blanking interval so as to illuminate the material 
during image capture periods . 

As will be appreciated from the above, capture of image 
data according to this aspect of the invention is based 
upon recognising the presence and position of the 
particular pattern model in a captured image during 
subsequent image processing rather than by triggering image 
capture upon the web achieving a particular position. The 
method may therefore be regarded as, and will be referred 
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to as, an "opportunist triggering" technique. In this way, 
the sub-pixel inspection method also facilitates the use of 
a normal video camera as opposed to a specialist 
(triggered) camera system. Image capture may take place at 
5 time intervals which can be externally monitored by looking 
at the camera SYNC signals and illumination of the web at 
the appropriate moments can be triggered by the SYNC 
signals. An advantage in performing image capture in this 
way, is that an ordinary video camera, as opposed to a 
0 higher cost image capture system, may be utilised. 
However, it will be appreciated that a triggerable camera 
may still be used with this method. 



Preferably, image data from the camera is analyzed so 
5 as to carry out a pattern recognition operation based upon 
the pattern recognition model to determine the physical 
position of the printed pattern repeat on the web in the 
field of view of the camera at the time of image capture. 

Following successful pattern recognition, analysis of 
the image captured to inspect the required areas displaced 
by the same displacements as- the pattern model may be 
carried out with sub-pixel accuracy. 

Preferably, the opportunist triggering technique is 
utilised in conjunction with the lighting non- uniformity 
compensating method which is also to be carried out to sub- 
pixel accuracy. 

Specific embodiments of the invention will now be 
described, by way of example only, with reference to the 
accompanying drawings, in which: 
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Figure 1 ^shows in schematic form a colour monitoring 
apparatus ; 

Figure 2 is a schematic graph showing camera output in 
5 volts, against input light intensity; 

Figure 3 is a schematic diagram showing a graduated 
grey scale "which may be incorporated within an image to be 
captured for ascertaining camera response; 

0 

Figure ^4 shows a black tile and a white tile forming 

part of a test image; 

Figure 5 illustrates how readings corresponding to the 
i black tile and white tile of Figure 4 may be used to 
determine a camera offset; 

Figure € is a schematic diagram showing possible 
relative size of repeat length with respect to size of 
field of view; 

Figures 7A to 7C show respectively first to third 
patterns to be printed by first to third cylinders and to 
be monitored using an opportunistic triggering technique; 

Figures ^ 8 and 9 s how respectively a training image and 
a newly captured image; 

Figures 10A and 10B show a perimeter area of a 
displaced image which it is desired to inspect to sub-pixel 
accuracy; and 
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Figure 11 shows a hardware arrangement featuring a 
scanner arranged to move a camera and flash in a transverse 
direction relative to a moving web. 

Referring to Figure 1 of the accompanying drawings, 
there is shown a typical set of apparatus which may be used 
for colour monitoring. The apparatus comprises an 

interface unit for interfacing to a conventional 
commercially available video camera 1, having a red, green, 
blue output (RGB output) and having a controllable iris 
aperture, a controllable overall gain, independently 
controllable RGB signal channel gains, or a controllable 
signal channel balance. The interface comprises an RS232C' 
bus 2 to enable remote control of the camera and a 
conventional commercially available personal computer 3 
having a conventional commercially available image capture 
board 4 . The camera 1 is positioned within a lighting box 
5 which also contains a flash lighting source 6. A trigger 
circuit 7 is provided which is connected to the camera 
output 1, to the lighting source 6 and to the image capture 
board 4 and to the PC 3 . A monitor 8 is also provided. 

The computer is adapted for control of the camera via 
the RS232C data bus, by means of a dedicated application 
programme. The dedicated application programme may form 
part of a larger computer programme, or may be run in 
conjunction with other application programmes. 

Aspects of the present invention are primarily directed 
towards improving the usability of such a system by 
addressing the problems of non-linearity of camera 
characteristic, non-uniformity of lighting of a specimen to 
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be monitored and accurately measuring the colour of non- 
uniform areas on the web. 

Concerning the first problem of non-linearity of camera 
5 characteristics, an embodiment of the first aspect of the 
present invention will now be described with reference to 
Figures 2 to 5 . 



10 



Figure 2 shows a typical camera characteristic in terms 
of increasing light intensity of a known colour (R, G or B) 
along the horizontal axis and measured R, G or B output in 
volts along the vertical axis. 

The graph of Figure 2 may be obtained, for instance, by 
15 capturing the image of a collection of paint chips of 
graduated colour, such as the grey scale 3 0 shown in Figure 
3. All paint chips must be captured in a single image 
capture period and under uniform illumination. 



H 20 The known colours can be measured using established 

colorimetric techniques, i.e. a colorimeter or 

spectrophotometer traceable to national colour measurement 

standards, coupled with a standard conversion matrix to 

convert the CIE tristimulus measurements into camera R, G, 
25 B values. 



Referring to Figure 3, the left most paint chip 31 is 
black, the right most paint chip 32 is white and, in 
between, are chips of intermediate shades of grey. The 
images of each of those chips are captured to plot the 
graph of Figure 2. Note that, the vertical axis may be any 
one of the measured R, G or B outputs and* also that in 
obtaining this graph the iris _of the camera l is set so 
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that the white chip produces a near saturation R, G and B 
output, the horizontal axis represents increasing light 
intensity from black to white, in known increments as 
determined by the test pattern of Figure 3 . 

5 

Referring to the characteristic plot of Figure 2, it 
can be seen that the graph is generally linear, but tends 
to flatten out above a knee point (K) . Also, there is an 
origin offset (OFF) . It will be seen that in the plot 

0 illustrated, the paint chip 32 representing the highest 
light intensity level, pure white (i.e. the most white) is 
shown as a plotted point 20 which falls within the non- 
linear region. The graph of Figure 2 also shows the next 
"nearest to white" paint chip as giving a plotted point 21 

5 which is in the non-linear region. 

In an ideal world, the camera characteristic would pass 
through the origin and be totally linear. However, as seen 
from the above, conventional video cameras do not operate 
0 in this ideal fashion and it is necessary to compensate for 
this non-linearity if accurate determinations of light 
intensity are to be made. 

In order to provide an effective compensation, the 
offset (OFF) a-nd the knee point (K) need to be determined. 

To complicate matters more, camera characteristics 
including the offset change by measurable amounts as the 
camera 1 warms up and cools down, and as ambient 
temperature changes. So for accurate colour monitoring 
purposes, compensation needs to occur by varying amounts 
every time an image is captured. 
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In order to determine the offset, a black object (zero 
reflectance) may be included in the image at each image 
capture point . By analyzing the output data from the 
camera relating to the black object area of the image, the 
5 average corresponding R, G and B output values may be found 
and those values can be directly taken as the offset values 
for the respective R, G and B channels. 

The drawback with this approach is that it is difficult 
0 to get a perfectly black object, and to make sure that it 
picks up no contaminant prior to being monitored that 
changes its colour. In view of this, a better approach is 
to use a white ceramic tile 40 (see Figure 4) of known RGB, 
and a black ceramic tile 41 of known RGB simultaneously 
5 present within the camera field of view 42 (the RGBs can be 
measured using a colorimeter or a spectrophotometer 
traceable to national colour measurement standards and by 
using a standard matrix for converting those measurements 
to camera RGB values) . Ceramic tiles may be easily cleaned 
by using lens tissue before image capture takes place. 

Prior to using this black and white tile method, it 
must be established that the camera is operating in a 
linear fashion. This may be done by forming the graph of 
Figure 2, restricting the size of aperture to avoid going 
beyond the knee, and then repeating the graph measurements 
to check that all of the points on the measured grey scale 
now produce R, G, B outputs within the linear region. All 
of the grey scale needs to be captured during a single 
capture period (i.e. from a single image) under uniform 
lighting conditions. Adjusting the voltage output to be a 
proportionate reduction of its full scale value when 
inspecting white portions of the image will in practice, 
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suffice in many practical situations as long as the whole 
field of view is uniformly lit or the white tile is in a 
sufficiently well lit part of the field of view that no 

part of the field of view produces above knee signals and 
5 may be used for all camera operations once it has been 

verified that the camera is indeed operating wholly within 

the linear zone. 

Having ensured that the camera is being operated below 
10 its knee point, a graph as shown in Figure 5 may be 
obtained from the measured white tile and black tile by 
drawing a line between the two known points and extending 
that line to find the offset from the origin. 

15 The two known points can be measured using established 

colorimetric techniques, i.e. a colorimeter or 
spectrophotometer traceable to national colour measurement 
standards and by using a standard conversion matrix for 
converting these measurements to camera R, G, B values. 

20 

If it is ensured that the black and white tiles are 
permanently in the field of view, then the calculation of 
offset (by using the two points given by black and white 
tiles) may be done by the computer each time an image is 
25 monitored -to provide reliable compensation for camera 
characteristics in real time situations. 

An alternative for checking the offset is to close the 
iris of the camera and use the R, G and B average values 
30 with the iris closed, to directly determine the offset. 



Turning now to the second problem of prior art systems 
which relate to non- uniformity of lighting characteristics 
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across the material to be monitored, preferred embodiments 
of methods for compensating such non-unif ormi ty will now be 
described. 

5 Non-unif ormi ties in lighting, particular 

characteristics (e.g. internal reflections) of the lighting 
enclosure from which individual images are captured, and 
physical characteristics of the substrate material can all 
cause anomalies in the amount of reflected light (or 

o transmitted light in the case of a back- lit web) arriving 
at the camera lens from different spatial locations within 
the camera field of view and need to be compensated for. 

The principle behind such a compensation technique is 
5 to capture an image of a uniformly coloured or unprinted 
sample of material . Spatial image data from the unprinted 
web and incident to the camera lens embodies data 
concerning not only the uniformity of incident rays in 
different parts of the field of view but also the 
translucency of the web and the internal reflections from 
the lighting enclosure. 

It is known, of course, that the unprinted sample of 
material is, in fact, an essentially uniform surface. By 
detecting spatial non-uniformities in camera output over 
the field of view, normalising settings may be achieved, so 
that individual output data from individual spatial areas 
is compensated. For example, the average R, G and B of an 
area of surface colour are recorded at its starting 
(trained) position, R T/ Gr, B T . When the image is next 
captured, suppose this area in the design has moved to a 
less well-lit position and its average R, G and B are now 
Rs/ G s , B s . If R: Gi, Bi, are the average RGB values for the 
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area of the imprinted web corresponding to the starting 
position, and R 2/ G 2 , B 2 the average values of the imprinted 
web corresponding to the less well -lit position, then R S/ 
G s , B s are normalized as follows before comparing them with 
5 the trained values R 7/ G?, B T : 

R s x Ri/R 2 , and the same for G s and B s . 

Concerning the third aspect, there will now be 
10 described how an opportunist triggering technique may be 
used to monitor a moving web of material without the need 
for complex triggering methods. 

Referring now to Figure 6, there is shown a typical web 
15 of material which it may be desired to monitor. Figure 6 
shows schematically, the nominal field of view seen by the 
monitoring camera and represented by rectangle 60 
superimposed over a schematic view of moving web of 
material 61 . 

20 

It will be appreciated that a pattern printed on the 
web 61 will have a given repeat length R. The repeat 
length defines the pattern printed on the web in its 
entirety and may be wider (as shown here) or narrower than 
25 a width W of the field of view 60 of the monitoring camera. 

In previous systems, if it was desired to monitor a 
full repeat of the web 61, or indeed to simply monitor 
known portions of the sampled web, a reference mark on the 
3 0 web could be used in conjunction with a sensor to trigger 
an image capture cycle when the web has reached a 
particular point. In such systems, image capture is driven 
by the positioning of the web, and the image capture system 
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must be ready to fire whenever the reference mark aligns 
with the triggering sensor. To implement such systems, the 
monitoring camera needs to be of a relatively sophisticated 
nature . 

Embodiments of the present invention tackle the problem 
of triggering in a different fashion. 

In embodiments of the present invention, a basic video 
camera may be used and image capturing driven by 
synchronisation (SYNC) signals from the camera itself. 

According to the method, a particular unique feature 
within the full repeat (R) has its image data pre-stored in 
memory, so that it may be used for pattern recognition 
purposes. With the web of material running, 

synchronisation signals output from the video camera are 
monitored and lighting and image capture from the web 
triggered in accordance with the synchronisation signals, 
so that whenever the camera itself is ready to perform an 
image capture operation, and such an image capture 
operation is called for, lighting of the web is 
automatically synchronised and carried out on the basis of 
the SYNC signals. 

Once an image capture operation has been performed, it 
is necessary to determine what the image is of . In that 
regard, the captured image data is analyzed using a pattern 
recognition programme, so as to determine exactly what the 
image is of, in relation to the unique feature in the 
pattern repeat. Thereafter, once the system knows exactly 
what the captured image data is of, standard colour 
monitoring procedures .may. be carried out to monitor qualitv 
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of printing etc. and utilising any other software 
compensation techniques with regard to non-linearity 
ccmpensation/non-unif ormity of lighting compensation etc. 
to determine meaningful results. 

5 

A typical mode of operation of the apparatus of the 
present invention will now be described. 

With unprinted web material present and with the 
10 aperture restricted (if required) to ensure operation in 
the linear range of the camera characteristic, an image is 
captured in order to perform the lighting uniformity 
compensation referred to earlier. This image is hereinafter 
referred to as a "uniformity image" 

15 

Thereafter, with the approved printed material running 
on the press and with white and black reference tiles 
present, an image is captured. The reference tiles serve 
to enable an accurate determination of slope and offset of 

2 0 the camera characteristic during each capture operation as 
described previously and the white tile also serves to 
control the camera and keep the colour measurements 
repeatable as described in PCT application WO96/05489. 
Rectangles may be drawn over particular areas of the 

25 pattern to be monitored. A sufficient number of images are 
captured, in order to record details of the full repeat. 

Next, a training operation is carried out. Training 
does the following: it automatically selects a pattern 
30 model and trains the colours, using lighting uniformity 
compensation and any other required software adjustments. 
The system may then be set to what is known as an "auto- 
inspect" mode, in which it will monitor the colours of the 
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areas defined by the rectangles in the training image at 
ten second intervals throughout the run, giving real-time 
and logged results. Auto-inspect does the following: 
captures an. image at intervals from the moving web, keeping 
the camera controlled; finds the pattern model and 
calculates image displacement from a trained image; 
inspects the coloured areas, allowing for the displacement 
and lighting uniformity; logs the results to file; and 
signals if the colour of any area is beyond warning limits. 

During such automatic inspection, the training image 
will be displayed at all times and the ongoing measurements 
of any defined area may be displayed at will. 

This opportunist triggering technique produces a 
distinct advantage over traditional web- triggering in 
applications where the print cylinders are not in register 
wvth each other. 

For example, in the printing of filter- tip cigarette 
paper where it is desired to print a cork design on the 
paper in a seemingly random fashion there is illustrated in 
Figures 7A to 7C a pattern to be printed by means of three 
cylinders . 

The first cylinder shown in Figure 7A is utilised to 
print a manufactures mark 7 0 (LOGO) . The mark (LOGO) needs 
to be printed once per cigarette. 

The second cylinder shown in Figure 7B is used to print 
a first colour 71, for example a solid yellow 11. The 
third cylinder shown in Figure 7C is utilised to print a 
further pattern 72, for instance a brown colour with gaps 
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formed in it to allow the yellow to show through those 
gaps. Prints 71 and 72 together form the cork design. 

The mark 7 0 must be printed on every cigarette, but the 
5 filter tip design, made up of the first colour 71 from 
cylinder 2 overlaid by brown from cylinder 3, must appear 
random, so its repeat is say 3.5 cigarettes. 

Each cylinder further prints a rectangular box 73-75 at 

0 the edge of the web showing its ink colour once per 
cylinder circumference as well as a text string 76, 11, 78. 

Because it is desired to produce a non- regular pattern 
there is no requirement for the cylinders to have the same 

1 circumference, nor for them to rotate in phase with each 
other. The ink boxes therefore appear at random relative 
to each other, and this causes problems in web -triggered 
inspection systems . 

Because it is the colour of the ink boxes which is to 
be monitored, and because each ink box is a fixed distance 
from its associated text string, the problem with regard to 
web -triggering which was encountered in the prior art may 
be removed altogether by the use of the opportunistic 
triggering techniques described previously. This is 

achieved by using the unique text string 76-78 which 
accompanies each ink box 73-75 as a pattern model, and 
searching can be done every captured image for all 3 text 
strings, to thus find the ink boxes 73-75 and then inspect 
their colour. 
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Referring now to Figures 8 and 9, there is illustrated 

a method for inspection of an area of a web of material to 
sub - pixe 1 accuracy . 

5 Figure 8 shows a camera field of view with a web of 

material 8 0 , on which there is printed a logo 81 and other 
image data 82 including • an inspection area 83. The view 
shown in Figure 8 is, for instance, an original training 
image used during setup. 

10 

Figure 9 is an image captured during a subsequent image 
capture cycle and featuring the same elements as the image 
shown in Figure 8. In Figure 9, it will be noted that the 
web has been displaced to the right of the field of view by 
15 an amount and that the actual positioning of the design is 
vertically displaced. This vertical displacement can occur 
due to triggering inaccuracy or may occur as a result of 
the use of an opportunistic type triggering method as 
previously described. 

20 

The element 81 referred to above as the pattern model 
whose data is stored from the original training image is 
used as a search pattern to locate the same pattern in the 
captured image. Once that design has been located in that 

25 captruxed image, its displacement can be calculated which, 
for instance, may be 50.6 pixels to the right and 73.3 
pixels down. Knowing the displacement of the pattern 
model, the area to be inspected 83 can also be located by 
using the same displacement, i.e., it will be known that 

30 that area is also 50.6 and 73.3 pixels displaced relative 
to_jLts original position. 
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In the newly captured image, a rectangle of pixels may 
be processed which is slightly larger in area than the 
original inspection rectangle in the training image and a 
margin of error given by adding an extra row and an extra 
5 column of pixels. Because the displacement involves pixel 
fractions, the area of the design to be inspected now 
straddles pixels. Here, the assumption is made that the 
digitised value of the voltage level corresponding to 0.1 
pixel corresponds to 0.1 of the digitized value of the 
10 voltage level for that entire pixel. In order to measure 
the average R, G or B values of the area, a fraction of the 
pixel values around the perimeter is taken as follows: 



Top left corner take 0.4x0.7 x R, G or B value of this pixel 
Top right corner take 0.6x0.7 x R, G or B value of this pixel 
Bottom left corner take 0.4x0.3 x R, G or B value of this pixel 
Bonom right corner take 0.6x0.3 x R, G or B value of this pixel 
Top row of pixels take 0.7 X R, G or B of all these pixels 
Bottom row of pixels take 0.3 X R, G or B of all these pixels 
Left row of pixels take 0.4 x R, G or B of all these pixels 
Right row of pixels take 0.6 X R, G or B of all these pixels 



.5 Add the R, G or B of each pixel in the displaced area, 

and the R, G or B of each straddling pixel, weighted as 
shown in the above table. Divide the grand total by (width 
x height) , and this gives the average R, G or B of the 
inspected area to sub-pixel accuracy. Each of the channels 

0 R, G and B must be processed independently. 
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To explain the above further, Figures 10A and 10B are 
referred to. 

Figure 10A shows a training image together with a 
5 inspection area 83 to be inspected and marked with a 

rectangle A (shown as ) . Each illustrated "square" 

represents a pixel and the inspection area necessarily 
encompasses whole (integer) pixels. Note that an 

unrealistically small inspection area of 5x5 pixels is 
LO shown here so as to enable pixel fractions to be 
demonstrated. Actual inspection areas used would in 
practice be very much larger in area. 

Figure 10B shows a captured image with the inspection 
.5 area 85 displaced by 50.6 pixels in the horizontal (x) 
direction and 73.3 pixels in the vertical (y) direction. 

For the purposes of the following discussion, it is of 
course evident that to illustrate the boundary conditions, 
an actual displacement of 50 x 73 cannot be shown in Figure 
0 10B. Rather, Figure 10B is used solely for the purposes of 
discussing what happens at the subpixel level . The square 
A containing inspection area 83 of Figure 10A is shown in 
Figure 10B as having moved to position A 1 . The "floor" 
integer pixel position, i.e. the pattern model shifted by 
5 50 pixels horizontally and 73 pixels vertically is marked 
as rectangle C (shown _•_-_.) . 

In the Figure, Ax = fractional pixel horizontal 
displacement (+VE meaning displacement to the right) ; Ay = 
) fractional pixel vertical displacement (+VE means 
displacement in a downward direction) ; x-displ = integer 
part of horizontal displacement; y-displ = integer part of 
vertical displacement. 
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In the example given, Ax = 0.6, Ay = 0.3, floor x-displ 
= 50, floor y-displ = 73. 

5 If Ax and Ay are positive, a column of pixels to the 

right of the displaced area and a row of pixels below the 
displaced area are processed. If Ax and Ay are, on the 
other hand, negative, a column to the left and a row above 
are processed. 

10 

The same equations for calculating the fraction of each 
pixel in the bottom, top, left and right rows and columns 
in the extended processing area are used whether Ax and/of 
Ay happen to be positive or negative (+VE or -VE) . In 

15 otherwords, concerning the perimeter area for the example 
shown in Figures 8/9 and the displacements of which are 
illustrated schematically in Figures 10A and 10B, the 
fractional pixel values providing the weighting as given in 
the table 1 are shown as shaded areas in Figure 10B and are 

20 obtained by performing the following calculations: 

(i) Top left hand corner pixel = (1-Ay) (1-Ax) x 
R, G or B pixel value 

(ii) Top right hand corner area pixel - (1-Ay) 
25 (Ax) x R, G or B pixel value 

(iii) Bottom left corner pixel = (Ay) (1-Ax) x R, G or 

B pixel value 

(iv) Bottom right corner pixel = (AY) (Ax) x R, G 
or B pixel value 

30 (v) Top row of pixels = (1-Ay) x R, G or B pixel 

value 

(vi) Bottom row of pixels = Ay x R, G or 3 pixel 

value 
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(vii) Lefc row of pixels = (l-£x) x R, G or B pixel 

value 

(viii) Right row of pixels = Ax x R, G or 3 pixel 
value 

5 

Again, when inspecting these perimeter values the 
assumption is made that the digitized value of the voltage 
level corresponding to 0.1 pixel corresponds to 0.1 of the 
digitized value of the voltage level for that entire pixel. 

10 

By referring to the above, it may be seen that there is 
provided a method for inspection of an area of a web of 
material to sub-pixel accuracy which facilitates web 
inspection even when parts of the design on the entire web 
15 appear to vary in a seemingly random fashion. 

Figure 11 shows a hardware arrangement with a camera 85 
and flash 86 associated with one another on a moving sled 
87 of a scanner 88. The camera 85, flash 86 and sled 87 of 
the scanner 88 are shown mounted underneath a moving web of 
material 89. A lighting enclosure 90 is also associated 
with the scanner 88. In the arrangement shown, the web is 
arranged to move from left to right as shown in the figure 
and the camera and flash are mounted for up and down 
movement . 

In operation, the camera and flash move transversely 
with respect to the web so as to provide a means of 
mechanical compensation for transverse ( u x"-axis) movement 
of the web. The amount of mechanical movement of the 
camera and flash is calculated by means of the above- 
mentioned procedure for measuring the displacement of a 
pre -stored pattern model in a captured image relative to 
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its position in a training image and translating (e.g. 
using a scaling factor) that displacement value into a 
u real -world" value by which the camera is moved so as to 

mechanically compensate by reducing said displacement. 
5 After movement of the camera 8 5 and flash 86, there is then 

performed a subsequent image capture operation in which the 

transverse movement of the web is already compensated to a 

degree . 

L0 Such mechanical compensation is not as accurate as that 

which may be achievable by sub-pixel interpolation as 
previously described, so in practice a combination of the 
mechanical method together with sub-pixel interpolation is 
preferred. In this combination method, following movement 
5 of camera 85/flash 86, pattern model displacement is again 
measured and the step of colour calculating then performed. 

When the camera 85 is a triggered camera, a variable 
delay can be introduced between the sensing of the trigger 
signal (the trigger signal being activated from a feature 
in the design repeat) and the initiation of the flash and 
image capture. This allows the "y" displacement to be 
compensated. Once again, this method is not as accurate as 
the sub-pixel interpolation method, so, . ideally, those 
methods are used in combination. 

The reader's attention is directed to all papers and 
documents which are filed concurrently with or previous to 
this specification in connection with this application and 
which are open to public inspection with this 
specification, and. the contents of all such papers and 
documents are incorporated herein by reference. 
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All of the features disclosed in this specification 
(including any accompanying claims, abstract and drawings) , 
and/or all of the steps of any method or process so 
disclosed, may be combined in any combination, except 
5 combinations where at least some of such features and/or 
steps are mutually exclusive. 



Each feature disclosed in this specification (including 
any accompanying claims, abstract and drawings) , may be 
.0 replaced by alternative features serving the same, 
equivalent or similar purpose, unless expressly stated 
otherwise. Thus, unless expressly stated otherwise, each 
feature disclosed is one example only of a generic series 
of equivalent or similar features. 

5 

The invention is not restricted to the details of the 
foregoing embodiment (s ) . The invention extends to any 
novel one, or any novel combination, of the features 
disclosed in this specification (including any accompanying 
0 claims, abstract and drawings), or to any novel one, or any 
novel combination, of the steps of any method or process so 
disclosed. 



